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Abstract 

Q_| ' We consider a system of nonlinear Schrodinger equations with 
<^ three waves interaction studying the existence of ground state som- 
en | tions. In particular, we find a vector ground state, namely a ground 
+^ ■ state (u\,U2, U3) such that Uj ^ 0, for all i = 1, 2, 3. 

^ : 1 Introduction 

> : 

When light is scattered from an atom or molecule, most photons are elasti- 
iD ■ cally scattered (Rayleigh scattering), such that the scattered photons have 

the same energy (frequency) and wavelength as the incident photons. A 
small fraction of the scattered light (approximately 1 in 10 million pho- 
; tons), however, is scattered by an excitation, with the scattered photons 

having a frequency different from, and usually lower than, the frequency 
of the incident photons. This process is know as Raman effect or Stimu- 
lated Raman Scattering. The Raman amplification is based on this phe- 
nomenon: when a lower frequency signal photon induces the inelastic 
scattering of a higher-frequency pump photon in an optical medium in 
the nonlinear regime, as a result of this, another signal photon is pro- 
duced, with the surplus energy resonantly passed to the vibrational states 
of the medium. This process, as with other stimulated emission processes, 
allows all-optical amplification. In addition to applications in nonlinear 
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and ultrafast optics, Raman amplification is used in optical telecommu- 
nications, allowing all-band wavelength coverage and in-line distributed 
signal amplification. For physics and engineering aspects about this topic 
see, for example, [9]. 

This phenomenon, in a suitable mathematical contest, has bee treated 
in [5, 6], where a new set of equations describing nonlinear interaction 
between a laser beam and a plasma has been derived. From a physics 
point of view, when an incident laser field enters a plasma, it is backscat- 
tered by a Raman type process. Then these two waves interact to create 
an electronic plasma wave. The three waves combine to create a variation 
of the density of the ions which has itself an influence on the three pro- 
ceedings waves. The system describing this phenomenon is composed by 
three Schrodinger equations coupled to a wave equation. 

Later on, this model, slightly modified, has been studied in [7, 8], where 
in particular the orbital stability of solitary waves is studied for the follow- 
ing system of nonlinear Schrodinger equations: 

id t v 1 = -Avx - - 7f 3 W2", 

id t v 2 = —Av 2 - \v 2 \ p ~ 1 v 2 - jv 3 W[, (1) 
id t v 3 = -Av 3 - |w 3 | p " 1 ^3 - 7^2, 

where v\, v 2 and v 3 are complex valued functions of (t, x) e R x M. N , N = 
1, 2, 3, 1 < p < 1 + 4/N, 7 > 0. More precisely, if <p e H l (R N ) is the unique 
positive radial (least-energy) solution of 

—Av + ojv — \v\ p ~ x v = inR N , 

with u > (see [2, 10]), the authors study the orbital stability of the follow- 
ing standing wave solutions: (e^V; 0, 0), (0, e luJt (p, 0) and (0, 0, e luJt (p). They 
show that (e^V, 0, 0) and (0,e luJt ip,0) are orbitally stable, for any 7 > 0, 
while (0, 0, e tuJt ip) is orbitally stable, if < 7 < 7*, and it is orbitally unsta- 
ble, if 7 > 7*, for a suitable positive constant 7* = 7*(iV, p, tu). 

If, moreover, among the solutions of (1), we look for standing waves, 
namely solutions of the type 

(v 1 (t,x) J v 2 (t J x) J v 3 (t J x)) = (e^ 1 *n 1 (x),e^ 2 *n 2 (x),e^n 3 (a;)), 

where u\,u 2 and w 3 are real valued functions of x e M. N , it is easy to see 
that, if CU3 = io\ + lu 2 , (e lu ' lt ui(x),e tu ' 2t u 2 (x),e luJ3t u-i(x)) is a solution of (1) if 
(tti, u 2 , u s ) is a solution of the following system: 



— Au\ + uj\U\ — 
-Au 2 + cu 2 u 2 - 
-Au 3 + u 3 u 3 - 



(2) 
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Therefore, the orbital stability of such type of solutions is strictly linked 
with the existence of a least energy solution of (2). 

Motivated by these previous results and by the above considerations, 
in this paper we are interested in a generalization of system (2). More 
precisely, we consider the following system: 



— Aui + ujiUi — luil^Ui = 7M2M3 in M. N , 

-Aw 2 + u) 2 u 2 - \u 2 \ p ~ 1 u 2 = inR-^, . > 

-Au 3 + - |u3| p ~ 1 u 3 = 7M1M2 in R N , 
ui,u 2 ,u 3 G if 1 ! 



where > 0, % = 1,2,3, 7 G R and 2 < p < (N + 2)/{N - 2). It is 
easy to observe that (V w ) possesses trivial solutions different from (0, 0, 0). 
Indeed, by [2] there exists a (least-energy) solution Ui G _fT 1 (IR Ar ) for the 
single Schrodinger equation 

-An + ujiu = \u\ p ~ l u in R N . 

It can be checked immediately that («i,0, 0), (0, u 2 , 0) and (0,0, u 3 ) are non- 
trivial solutions of (Vuj). We will refer to these solutions as scalar solution 
and, of course, they are meaningless. We are looking for other type of 
solutions that we will call vector solution. 

We set u = («!, u 2 , u 3 ) and H = H\R N ) x if 1 (R JV ) x fl 1 ^). 

To fix terminology, we introduce the following definition. 

Definition 1.1. A solution of (VJ), u G HI, u ^ (0, 0, 0) wz'ZZ be caZZerf sozkr 
solution if there exist i, j — 1, 2, 3, z ^ j, swc/t rTzaf = u,,- = 0; w/tz'Ze a solution 
u G HI 0/ (Pa;) wz'ZZ be azZZed vector solution ifui ^ 0,for i = 1,2, 3. 

System (7^) has a variational structure and so its solutions can be found 
as critical points of the functional I : H — ► R defined as follows: 

3 

J(u) = 22 Ii(ui) - 7 / M1M2M3, (3) 

where, for z = 1, 2, 3, 

^i(wi) = t: / |Vu»| 2 + — / u P+] 



^ JR N P 

We are interested in the existence of a ground state solution, namely u G 
H, u ^ (0, 0, 0) which solves and minimizes the functional I among 
all possible nontrivial solutions. 

As a first step, we prove that, for any 7 G R, the problem (P u ) admits a 
ground state. 



4 



A. Pomponio 



Theorem 1.2. For any oji > 0, i = 1,2,3 and for any 7 e R, t/zere exists 
a ground state solution u G Ho/ (7^)- Moreover 1^ is a radially symmetric 
function (up to translation), for i = 1,2, 3. 

Then we prove that vector solutions exist whenever the absolute value 
of the coupling parameter 7 is sufficiently large. 

Theorem 1.3. For any Ui > 0, i = 1,2,3, there exists 70 > such that, for any 
7 6 M with I7I > 7 0/ (V w ) possesses a vector solution u e M, which is a ground 
state solution. 

The main result of the first part of this paper is Theorem 1.3: up to 
our knowledge, indeed, this is the first vector solution existence result for 
problem (VJ). Theorem 1.2, instead, is essentially already known. The 
symmetry result is due to [4] while the existence result is proved in [3]. 
For the reader's sake, here we give a different proof of the existence of a 
ground state and our arguments are based on the constrained minimiza- 
tion over the Nehari manifold. Theorems 1.2 and 1.3 will be proved in 
Section 2. 

By a mathematical point of view, system (Vw) is related with weakly 
coupled nonlinear Schrodinger systems (see for example [14] and the bib- 
liography therein). However, the peculiarity of (V u ) is due to the pres- 
ence, in the functional / defined in (3), of an integral term without a pre- 
assigned sign and this, of course, generates several difficulties. 

In the second part of the paper, we try to generalize further on system 
(PJ). We consider, indeed, the case when we substitute in (VJ) the positive 
constants uji with non-constant positive potentials Vi(x). More precisely, 
we consider 

—Ami + V\(x)u\ — |wi| p-:1 "Ui 
-Au 2 + V 2 (x)u 2 - \u 2 \ p ~ 1 u 2 
-Au 3 + V 3 (x)u 3 - Kl^us 
«i,m 2 ,«3 e H\R N ), 

where 7 G R, 2 < p < (N + 2)/(N — 2) and we assume the following 
hypotheses on V = (Vi, V 2 , V3): 

(VI) for all? = 1, 2, 3, Vi : ~§i N — > R is a measurable function; 

(V2) for all % = 1,2,3, V i:00 := lim^^oo Vi(y) ^ Vi(x), for almost every 
x E M. N , and the inequality is strict in a non-zero measure domain; 



7M2M3 in R , 
7U!U 3 inR^, 

r )U\u 2 inR^, l/W 



(V3) for all i = 1, 2, 3, < d ^ ^(2), for all x e R N . 
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Such type of assumptions has been introduced in [15] for the study of a 
single nonlinear Schrodinger equation. 

With concentration-compactness arguments, we prove the following: 

Theorem 1.4. Suppose that V = (Vi, V 2 , V 3 ) satisfies (Vl-3), then for any 7 G 
K, there exists a ground state solution u G M of (Vy)- 

As in the first part of the paper, we prove that vector solutions exist 
whenever the absolute value of the coupling parameter 7 is sufficiently 
large. In this case, moreover, we can also slightly weaken the assumptions 
on the potential V, in particular, instead of (V2), we make the following 
assumption: 

(V2') for all i = 1,2,3, V i)00 := lim^-,.^ Vi(y) ^ Vi(x), for almost every 
x G R N , and there exists at least one i = 1,2,3 such that the inequality 
is strict in a non-zero measure domain. 

We remark that (V2) requires that all the potentials V have the same geo- 
metrical behavior and in particular all the three potentials have to be non- 
constant; at contrary, the assumption (V2') is satisfied if, for example, only 
one of the three potentials is non-constant and with the right geometry, 
while the other two could be positive constants. 
The following theorem holds: 

Theorem 1.5. Suppose that V = (Vi, V 2 , V 3 ) satisfies (VI), (V2') and (V3), then 
there exists 70 > such that, for any 7 G ffi with \*y\ > 7 , (TV) possesses a 
vector solution uel, which is a ground state solution. 

Up to our knowledge, these last two theorems are the first results for a 
system of nonlinear Schrodinger equations with three waves interaction in 
presence of non-constant potentials. Theorems 1.4 and 1.5 will be proved 
in Section 3. 

Notation 

• If r > and x G R N , B r (x ) := {x e R N : \x - x \ < r). We denote 
with B r the ball of radius r centered in the origin. 

• We set u = (ui, u 2 , u 3 ), u n = (ui, n , u 2 , n , u 3 , n ) and V = (Vi, V 2 , V 3 ). 

• We denote by || ■ || the standard norm of i/ 1 (M Ar ). 

• We set H = H\R N ) x H 1 ^ 1 *) x H 1 ^ 1 *) and, for any u G H, we set 

H 2 = £- =1 INI 2 - 
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• For any 1 ^ s ^ +00, we denote by || ■ || s the usual norm of the 
Lebesgue space L S (R N ). 

• By Ci and q, we denote generic positive constants, which may also 
vary from line to line. 

• By o n (l) we denote a quantity which vanishes to zero as n — »■ +00. 



2 The constant potential case 

We prove Theorem 1.2 by a constrained minimization over the Nehari 
manifold. Let us define G : H — > E such that 

G(u) = /'(u) [u] = 52 J i M - 37 / 

1=? J ^ N 

= / ivwii 2 + - h p+i - 37 / 

~~[ ./R^ JR N 

then, any critical point, u, of / satisfies the following equality: 

G(u) = 0. 

We denote by J\f the so called Nehari manifold of I, namely 

Af:= {uGi\ {(0,0,0)} G(u)=o}. 

The next lemmas give useful informations on the Nehari manifold. 

Lemma 2.1. For any 7 e M, fhere exzsfs a positive constant C 7 , sizc/z that for all 
u e jV, || xj. || ^ C 7 . 

Proof The conclusion follows immediately, observing that, for any u G 
A/", we have 



|u|| 2 = Xlll^ll 2 ^ C Y1 / |Vui| 2 + c^ 
i=i i=i ^ RJV 

= C(V/ |M i r 1 -37/ W1W2W3 



, i=i 
3 



< C I 52 IHIp+1 + 3 l7l • 1 1 '"'0. 1 1 3 1 1 tfcSS 1 1 3 1 1 1*3 1 
\i=l 

^ c feikir +1 + 3 i 



171 • «i R2 F3 



. i=i 
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Lemma 2.2. M is a C 1 manifold and it is a natural constraint for the functional 
I, namely each critical point of 1^ is a critical point of the unconstrained func- 
tional I. 

Proof Let u e M, we have: 

3 

G'(u)[u] = V / 2|V^| 2 + 2^ 2 -Go+l)H P+1 -9 7 / Ul u 2 u 3 
~l Jr n Jr n 

3 

= V / -| Vw 4 | 2 - u) % u\ - (p - 2)|w 4 | p+1 < -C||u|| 2 < -C 7 < 0. 
Hence we get the conclusion. □ 



Lemma 2.3. For any u 6 E, u / (0, 0, 0) there exists a unique number i > 
such that iu e Af and 

I(iu) = max 7(iu). 
Proof Let uGl,u / (0, 0, 0). For any t > 0, we set 

f(t) = I(tu) = ^ - - Ct\ 



where 



A= s jT I | V«i| 2 + Uiu] > 0, 

3 

B =y. i^i p+i >o, 

C — 7 / U1U2U3 E R. 



We have to show that / : R + — > R admits a unique maximum. Let us 
observe that 

lim M = ^ >0 , lirn f(t) = -oo, 
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so there exists at least a t > such that 

/(t) = ngx/(f). 

Since any maximum point of / satisfies 

fit) = At- Bt p - 3Ct 2 = 0^A = Bt p ~ l + 3Ct, (4) 

we conclude if we prove that t is the unique solution of (4). 
We have to distinguish two cases. If C ^ 0, then it is easy to see that (4) 
admits a unique solution. Suppose, instead, that C < and set V = —C > 0. 
For t > 0, we set 

g (t) = ^ = A- Bt^ 1 + 3Vt. 

Let t = min{£ > | g(t) = 0} and suppose, by contradiction, that there 
exists ti> t such that 

g(t ) = g(h) = 0. (5) 
Since lim t ^ g(t) = A> 0, then g'(t ) ^ and so 

3V ^(p-l)Bt p - 2 . (6) 

Moreover by (5), there exists t 2 G (to, ti), such that ^'(^2) = 0. Hence 

3V=(p-l)Bf 2 - 2 >(p-l)Bt p - 2 , 

and we get a contradiction with (6). □ 

According to the definition of [11], we say that a sequence {u n } n van- 
ishes if, for all r > 

lim sup / u\ + u\ + u\ = 0. 

Lemma 2.4. Any bounded sequence {u n } n c M does not vanish. 

Proof Suppose by contradiction that {u n } n vanishes, then, in particular 
there exists f > such that 

lim sup / u 2 in = 0, for all i — 1,2, 3. 



n— >+oo 



Then, by [12, Lemma 1.1], we infer that u^ n — > in L S (R N ), i = 1,2, 3, for 
any 2 < s < 2*. Since {u n } n C A/", we have that u n — > in H, contradicting 
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Lemma 2.1. □ 

Now we are ready to prove Theorem 1.2. 
Proof of Theorem 1.2 It is easy to see that for any u e J\f we get 

hence, by Lemma 2.1 and since p > 2, we infer that 

m = inf I(u) > 0. 

We have to show that this infimum is achieved as a minimum. 
Let {u n }„ c TV be a minimizing sequence. By standard arguments (see 
[17]), we can suppose that {u n } is a Palais-Smale sequences for / at level 
m, namely 

7(u n ) — > m, as n — > oo, (8) 
i"'(u n ) — >• 0, as n — > oo. (9) 

By (7) and (8), we know that {u n } n is a bounded sequence in EL Therefore, 
by Lemma 2.4, {u n } n does not vanish, namely there exist C, r > 0, C 
R N such that 

/ U ln + ^2,n + ^3,n > C for all 71 > 1. (10) 

Due to the invariance by translations, without loss of generality, we can 
assume that £ n = G M^, for every n. 

Since {u n } n is bounded in H, there exist U\, u 2 , u 3 G i? 1 (M. JV ) such that, up 
to a subsequence, for i = 1, 2, 3, 

u i>n ^u h mL s loc (R N ), l^s<2*. 
Hence, by (9), we infer that I'(u) = 0. 

By (10), moreover, we can argue that there exists i = 1, 2, 3 such that it, ^ 0, 
namely u ^ (0, 0, 0), and so we can conclude that u e Af. By the weak 
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lower semicontinuity, we get 

™<^)=t/ E J(iv«.i 2 ^^) + 3^kr 

^ f 1 2 
< liminf ^ / - {\Vu hn \ 2 + Wi «J B ) + J? ' \u isn \ p+1 

= liminf 7(u n ) = m, 

n— >+oo 

hence u is a ground state for the problem (VJ). 

Finally, by [4], we infer that u; t is a radially symmetric function (up to trans- 
lation), for i — 1, 2, 3. □ 



Let us now prove the existence of vector ground state. Here we use 
some ideas of [13]. 

Proof of Theorem 1.3 We start proving that there exists 70 > such that 
(Voj) admits a vectorial ground state, for any 7 > 70- 
Let Ui G if 1 (M Ar ) be the positive radial ground state of 

-An + luiu = in R N , it G H\R N ). 

We conclude if we show that there exists u G JV such that 

/(u) < min (1(^,0,0), /(0,u 2 ,0), 1(0,0, u 3 )} = min /<(«<). (11) 

By Lemma 2.3, we know that there exists t 7 > such that £ 7 («i, u 2 , u 3 ) G 
A/", namely 



3 /t 3 n n 

V / I Vu t \ 2 + UiU 2 = t^ 1 J2 / \ui\ p+1 + 3t 7 7 / 
~l Jr n ~[ Jr n Jr 1 



Since the last integral is strictly positive, it is easy to see that 



lim t 7 = 0. (12) 

7— >+oo 



Moreover since 



r t 2 (v- 2)t p+1 

j(t y (u 1) « 3) «3)) = X; / 7T (|v^| 2 + ^ 2 )+ o, rr^ l^r 1 ' 

by (12), for 7 positive and sufficiently large, (11) is satisfied. 

Let us now prove the same conclusion also for 7 negative and sufficiently 
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large in modulus. 

By Lemma 2.3, we know that there exists i 7 > such that t 7 (— u 1; u 2 , u 3 ) £ 
A/", namely 

V / \Vui\ 2 + c^ 2 = tr 1 y / - 3t 7 7 / 

~T •/ K-^ ~T JR^ JR 



3t 7 7 / U1U2U3. 



i=i " m i=i 
Observing that 



lim t 7 = 0, 

7—*— 00 



we conclude arguing as in the previous case. □ 



3 The non-constant potential case 

In this section, in not stated differently, we will always assume that V 
assume (Vl-3). 

Solutions of (Vy) are critical points of the functional I\>: M. — > M so 
defined: 



J v (u) = yi Vi (ui) - 7 / u x u 2 u 3 , 
where, for % = 1, 2, 3, 

1 JM.N P+1 J R N 

As in the first part of the paper, let us define 

3 

g v (u) = / v ( u M u ] = y J iOi)M - 37 / M1M2M3 

3 

= y I |V^| 2 + V^u] - \ Ui \ p+1 - 3 7 / u lU2 u 3 , 
~[ Jr n Jr n 



then, any critical point, u, of Iv satisfies the following equality: 

G v (u) = 0. 

We denote by A/V the Nehari manifold of I v , namely 

JV V := {u£ H\ {(0,0,0)} G v (u) = 0}. 
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The following lemmas describe some properties of the Nehari manifold 
A/V- The proofs are very similar to those of the constant potential case and 
we omit them. 

Lemma 3.1. For any 76I, there exists a positive constant C 7 , such that for all 
u g A/v, ||u|| ^ C T 

Lemma 3.2. A/V *s a C 1 manifold and it is a natural constraint for the functional 
/v. 

Lemma 3.3. For any u 6 H, u / (0, 0, 0) there exists a unique number i > 
such that iu e A/V and 

I\(iu) = maxlv(tu). 
For all V, we assume the following definition 

c v := inf J v (u), 

uGA/v 

so that our goal is to find u £ A/V such that Jv(u) = c v , from which we 
would deduce that u is a ground state solution of (Py). 

Let us recall some preliminary lemmas which can be obtained by using 
the same arguments as in [15] (see also [1]). 

As a consequence of the Lemma 3.3, we are allowed to define the map 
t : H \ {(0, 0, 0)} -> M+ such that for any u G H, u ^ (0, 0, 0) : 

Iv(t u u) = maxlv(iu). 

Lemma 3.4. The following equalities hold 

cv = inf max I\r(q(t)) = inf max Jv(tu), 

sertG[o,i] u^(o,o,o) t>o 

where 

r={ ff GC([0 ) l] ) l) I (7(0) = (0,0,0), 7 V (^(1))<0, (7(1)^(0,0,0)}. 
Lemma 3.5. Lei {u n } n C EI, such fhaf ||u„|| ^ C > 0,/or ah n ^ 1, and 

max J v (tu„) ^ c v + S n , 

with 5 n — > + . Then, fhere exzsf a sequence {y n } n C and too positive num- 
bers R, /i > such £hai 

liminf/ + < + 

n J B R (y n ) 
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Lemma 3.6. Let {u„} n C Wand {t n } n c M+ such that < C\ ^ ||u n || < C 2 
and 

Iv(t n u n ) = max Jv(tu n ) — > cv, as n oo. 
T/zen fe sequence {t n } n possesses a bounded subsequence in E. 
Proof We have 

3 



°>y f \v Ui , n \ 2 +v l ( x )u 2 ir 

= tn I tP n y I \Ui^ n \ P 



+ 3 7 / 

Jr n j 

The conclusion follows from Lemma 3.5. □ 



Lemma 3.7. Suppose that Vi, V^ n G L co (R N ),for i — 1, 2, 3 and for all n ^ 1. 
I/H„ -> V$ m L°°(]R Ar ) fen c Vn -> c v , w/zere V = (^,1/ 2 ,1/ 3 ) and V n 

(^l,n,^,n,^3,n). 

Let Jqo : HI — »■ K be the functional so defined: 

3 



7oo(u) = y~]li,oo(Ui) - 7 / Ml 

7^ ^ 



«2«3, 



where, for i = 1, 2, 3, 



Ii.aa(Ui) = - [ IVWil 2 + Vi_r^(x)u 2 — / 



v + 1 

Moreover, if we set Voo = (Vi >00 , V 2i00 , V 3t00 ), we denote 

Coo := c Voo - 

As in [15], we have 

Lemma 3.8. IfY satisfies (Vl-3), then, for any 7GR, we get cv < c^. 

Proof By Theorem 1.2, there exists u G H a ground state solution of the 
problem 

-Aui + Vi t00 ui — \ui\ p ~ l ui = ^u 2 u 3 in M. N , 
-Au 2 + V 2>00 u 2 - \u 2 \ v ~ x u 2 = 7M1M3 in k7V , 
-Au 3 + V 3i00 u 3 - |it 3 | p_1 it 3 = 7M1U2 in K^, 



Ml, U2, U 3 G if 
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Let t u > be such that t u u G Af v . By (V2), we have 

Coo = -foo(u) > -foo(tuU) 

3 

= J V (* U U) +tlT2 i V i,°° ~ V i( X )) u2 i > C V, 

and then we conclude. □ 



3.1 Proof of Theorem 1.4 

Let {u n } n c Afy such that 

lim J v (u n ) = cy. (13) 

n 

We define the functional J: HI — > K as: 

^)-t/ E „5(|V"-l 2 ^(-K)H-^l". 



p+1 



Observe that for any u G A/v, we have Iv(u) = J(u). 
To prove Theorem 1.4, we need some compactness properties on the se- 
quence {u n } n . 

We denote by u n the measure 

^ = E J n l^ 2 + + 3§TT) (14) 

By (13) we have 

V n {R N ) = J(Un) C V 

and then, by P.L. Lions [11], there are three possibilities: 
vanishing : for all r > 



lim sup / du n = 0; 

dichotomy: there exist a constant c G (0,c v ), two sequences and 
{^n}n/ with r n — > +oo and two nonnegative measures ^ and v\ such 
that 

0<^ + ^<v n , -c, u 2 n (R N )^c v -c, 

supp(^) C B r „(£ re ), supp(^) c R N \ B 2rn (Q; 
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compactness : there exists a sequence {£„}„ in M, N with the following prop- 
erty: for any 5 > 0, there exists r = r(S) > such that 




dv n ^ cv — S. 



Arguing as in [16], we prove the following 

Lemma 3.9. Compactness holds for the sequence of measures {u n }n, defined in 
(14). 

Proof Vanishing does not occur 
Suppose by contradiction, that for all r > 

lim sup / du n = 0. 
In particular, we deduce that there exists f > such that 




Arguing as in Lemma 2.4, we get a contradiction. 

Dichotomy does not occur 

Suppose by contradiction that there exist a constant c e (0, c v ), two se- 
quences {in} n and {r„} n , with r n — > +oo and two nonnegative measures 
z/i and such that 

SUpp(^) C B rn (in), SUpp(^) C \ B 2rn (Cn). 

Let p n e C\R N ) be such that p n = 1 in B rn (Q, p n = in R N \ B 2rn {£ n ), 
< p n < 1 and | Vp n | ^ 2/r„. Set u i)Tl = pu ijn and w Ln = (1 - /o)u i)Tl , and we 
denote v n = (v 1>n , v 2jH , v 3>n ) and w n = (w 1>n , w 2 , n , w^ n ). 
It is easy to see that 

liminf J(v n ) ^ c, 

n 

liminf J(w n ) ^ Cv — c. 

Moreover, denoting ft„ := B 2r . n (in) \ B rn (£ n ), we have 

z/ n (f2„) — >• 0, as n — > oo, 
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/ |Vw iin | 2 + Vi{x)u 2 i n -^0, for all i = 1,2,3, (15) 

/ \ui, n \ p+l ->0, for all i = 1,2,3. (16) 
Moreover by (16), we have 

Ul,n«2,n«3,n ~> 0. (17) 



By simple computations, by (15), (16) and (17) we infer that for alH = 1,2,3 
|V^,„| 2 + Vi{x)vl n 0, / |V«v| 2 + K(x)™ 2 n -> 0, 

r,. _> (1 / \ w , IP+ 1 n 

l - / i,n| u i I \ UJ i,n\ w i 

i2 n J Qri 



Vl, n V2,nVz,n ~> 0, / Wl,n^2,n^3,n " > 0. 

Hence, we deduce that for alH = 1,2,3 

/ \Vu hn \ 2 + V l {x)ul n = I \Vv i>n \ 2 + V i {x)vl n 
Jr n Jr n 

+ / |V^, n | 2 + ^(xK 2 „ + 0n (l), (18) 

[ \u ijn \ P+1 = [ M P+1 +/ Kn| P+1 + On(l), (19) 

Jr n Jr n Jr n 



Ui, n U 2jn U 3tn = / V 1>n V 2jn V 3tn + / W ltn W 2 ,nW 3tn + O n (l). (20) 
./R^ Jr^ 

Hence, by (18) and (19), we get 

J(u„) = J(v n ) + J(w n ) + 0„(1). 

Then 

cv = lim </(u n ) ^ liminf J(v n ) + liminf J(w„) ^ c + (cv — c) = cy, 

ran n 

hence 

lim J(v„) = c, (21) 

n 

lim J(w n ) = cv — c. 
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Let us observe, moreover, that by (18), (19) and (20), we have 

= G v (u n ) = G v (v n ) + G v (w n ) + o B (l). (22) 
We have to distinguish three cases. 
CASE 1: up to a subsequence, G v (v n ) ^ 0. 

By Lemma 3.3, for any n > 1, there exists 9 n > such that 8 n v n 6 A/y, and 
then 

3 

J2 I \ VV ^ + - P rT l Kn\ P+1 = 3 7 ^n / Ul.^n^n- (23) 

By (23) we have 

3 

E / (*« " ^(l^l 2 + vfaKn) + (AT 1 - ^)k*l p+1 < o, 

and, by (V3), we deduce that 9 n ^ 1. Therefore, for all n ^ 1, by (V3) and 
(21), 

cv < W(0 n v n ) = J(9 n v n ) < J(v n ) -> c < cv, 
which is a contradiction. 

CASE 2: up to a subsequence, G v (w n ) ^ 0. 
We can argue as in the previous case. 

CASE 3: up to a subsequence, GV(v n ) > and Gv(w„) > 0. 

By (22), we infer that Gy(v„) = o„(l) and G v (w„) = o n (l). 

Since {u„} n c A/V, by (13) and by Lemma 3.5, we infer that there exists 

fi > and n G N such that 

3 

/ \ui,n\ p+1 > H, for n ^ no, 
and hence, by (19), there exists n^N such that 



3 



2 



E / Kn| P+1 > 9' ° r E / > £ fOT 72 ^ "I" 



18 



A. Pomponio 



Without loss of generality, suppose that 

3 



E/ l^r 1 > f , forn ^ 1. (24) 



i=l 



Let {#„} n be such that 9 n v n e A/v- Combining (18) and (24), we deduce 
that {9 n } n is bounded. If 9 n ^ 1 + o n (l), we can repeat the arguments of 
Case 1. Suppose, therefore, that 

lim 9 n — 9 > 1. 

n 

We have 

3 r 

37/ 



o n (l) = G v (v„) = V / |V?v| 2 + 7i(z)i;? n - - 3 7 / 



3 

p+1 



and so 

I V*v| 2 + K(xk> = o n (l), for i = 1, 2, 3, 



which contradicts (24). □ 

Proof of Theorem 1.4 Let {u„}„ be a sequence in A/V such that (13) 
holds. 

By (V3) and (13), we deduce that {u n } n is bounded in H, so there exists 
u 2 , u 3 ) e HI such that, up to a subsequence, 

Ui, n u t weakly in H 1 (R N ) , (25) 
u itn ^Ui in Lf oc (R 7V ), with 1 < s < 2*. (26) 

We define the measures {u n } n as in (14); by Lemma 3.9 there exists a se- 
quence {£ n }„ in with the following property: for any 5 > 0, there exists 
r = r(<5) > such that 

g^J (|v „ (|2+ ^ K) + _^_ kr< , (27) 



CLAIM: {£ n }n is bounded in WL N . 

Suppose by contradiction that, up to a subsequence, |£ n | — > oo, as n — > oo. 
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Fix n > and let = (V 1>ao - fi, V 2i00 - /i, V 3i00 - //) and 1^ the func- 
tional associated with Voo )At . For any n > 1, let z n = (zi jn , Z2, n , ^3,n)/ where 
^i,n = Wj, n (- — £„), and £ n > such that the functions t n z n are in the Nehari 
manifold of ioo,^ 

Let S > and consider r > such that (27) holds. For n sufficiently large, 
we have 

ViO + £n) ^ Vi )00 - /i, for all x G B r . 

Hence we have 

C V + O n (l) = J V (u n ) ^ / V (*nU„) 

^ C Voo , M + f E / WO* + ^n) - (^oo - A*)) *£n 
i=l 

+ f E / (^+e.)-(Hoo-//))4 

z • i J B c 

1=1 r 

> C V~„ - l E / ^ + - - A*) I Z tn- 

1 i=l ^ 

Since by (27) 

3 

E / + - (^.°° - I ^ ^ C5 ' for an y n ^ x ' 

i=i 

and {t„}„ is bounded (the proof is the same as in Lemma 3.6), we get that 
cv ^ — C5. By the arbitrariness in the choice of 5 > 0, we have 

c v ^ c Voo, M - Using Lemma 3.7 we conclude that c v ^ 0^, which contra- 
dicts Lemma 3.8. 

So {£ n }n is bounded in R-^ and then, by (27), for any 5 > there exists 
r > such that 

H^nll-H" 1 ^) < uniformly for n ^ 1 , i = 1,2,3. (28) 

By (25), (26) and (28), we have that, taken s e [2, 2*[, for any 5 > there 
exists r > such that, for any n ^ 1 large enough 

||Wj, n — Wj||i S ( R JV) ^ ||lt i)Tl — Wi 1 1 i» (B r ) + ll^n — Wj||i S ( B c) 

^ <5 + C II Wi,n||_ffi(B{!) + ||Wi||fl-l(B=) ) ^ (1 + 2C)5, 
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where C > is the constant of the embedding H x {Bf} ^ L S (B$). We 
deduce that 

Ui, n -> iii in L a (R*), for any s e [2, 2*[. (29) 

By (13), we can suppose (see [17]) that {u n } n is a Palais-Smale sequence for 
/v|at v anc t as a consequence, it is easy to see that {u n } n is a Palais-Smale 
sequence for J v . By (25) and (29), we conclude that I v ( u ) = 0- 
We have to show that u (0,0,0). Suppose by contradiction that u = 
(0, 0, 0), since {u n } n is in Av and by (29), we should have that ||u„|| — > 
and this is impossible by Lemma 3.1. Hence u ^ (0, 0, 0) and so u e J\fy. 
Finally, by (13), (25) and (29) and by (V2-3) we get 

c v < -^v(u) < liminf / v (u n ) = Cy, 

so we can conclude that u is a ground state solution of (7V). □ 



3.2 Proof of Theorem 1.5 

In this section, we assume that V satisfies (V2') instead of (V2). We start 
proving the analogous of Lemma 3.8. 

Lemma 3.10. If V satisfies (VI), (V2') and (V3), then there exists 70 > such 
that, for any 76M with \^\ > 70, we get cv < Ox,. 

Proof By Theorem 1.3, exists 70 > such that, for any 7 6 1 with I7I > 
7o, there exists u = (ui,u 2 ,u 3 ) G HI a vector ground state solution of the 
problem 

—Aui + V 1>00 ui - luil^ui = ^u 2 u 3 inE^, 
— Au 2 + V 2 ,ooU2 - \u2\ p ' 1 U2 = IU1U3 in R N , 
-A« 3 + V 3j0O u 3 - |^ 3 |^~ 1 ^ 3 = jmu 2 in R N , 

Ul,li2,M3 £ H\R N ). 

Let t u > be such that t u u e A/V- By (V2') and since u is a vector ground 
state and so 7^ 0, for alii = 1, 2, 3, we have 

Coo = -foo(u) ^ Joo(tuU) 

3 . 

= I v (f«u) + ^ V / (V i)O0 - Vi{x))v% > c v , 



and then we conclude. 



□ 
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Let us now prove the existence of vector ground state. 
Proof of Theorem 1.5 By Lemma 3.10, we infer that for |7| sufficiently 
large c v < c^. We can repeat the arguments of the proof of Theorem 1.4 
to prove the existence of a ground state u of (Vy)- Therefore, we have only 
to show that u is a vector solution. 
By [15], there exists a ground state, Ui G if 1 (IR 7V ), of 

-Au + V,(x)u=\u\ p - 1 u, mR N , ueH 1 ^), 

and moreover, it is easy to see that ui > 0. 

Now the proof is similar to that of Theorem 1.3 and so we omit it. □ 
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